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Estimation of Bit and Frame Error Rates of 
Low-Density Parity-Check Codes on Binary 

Symmetric Channels 

Abstract 

A method for estimating the performance of low-density parity-check (LDPC) codes decoded by 
hard-decision iterative decoding algorithms on binary symmetric channels (BSC) is proposed. Based on 
the enumeration of the smallest weight error patterns that cannot be all corrected by the decoder, this 
method estimates both the frame error rate (FER) and the bit error rate (BER) of a given LDPC code with 
very good precision for all crossover probabilities of practical interest. Through a number of examples, 
we show that the proposed method can be effectively applied to both regular and irregular LDPC codes 
and to a variety of hard-decision iterative decoding algorithms. Compared with the conventional Monte 
Carlo simulation, the proposed method has a much smaller computational complexity, particularly for 
lower error rates. 

Index Terms 

Low-density parity-check (LDPC) codes, finite length LDPC codes, iterative decoding, hard-decision 
decoding algorithms, binary symmetric channels (BSC), error floor. 

I. Introduction 

Low-density parity-check (LDPC) codes, which were first proposed by Gallager [1], have 
attracted considerable attention over the past decade due to their capacity-achieving error per- 
formance and low complexity of the associated iterative decoding algorithms. Iterative decoding 
algorithms are particularly simple if binary messages are used. Such algorithms, which are 
referred to as hard-decision iterative algorithms, are the subject of this paper. Examples are the 
so-called Gallager algorithms A (OA) and B (GB) [1], [2], [3], their variants [4] and majority- 
based (MB) algorithms [5]. 
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The asymptotic performance analysis of LDPC codes under iterative decoding algorithms is 
now a rather mature subject. On the other hand, the finite-length analysis of LDPC codes is still 
an active area of research with many open problems. In this paper, our focus is on practical 
LDPC codes with finite block lengths in the range of a few hundreds to a few thousands of bits. 

In [6], Di et al. studied the average performance of the ensembles of regular finite-length 
LDPC codes over binary erasure channels (BEC), by relating the decoding failures of the belief 
propagation algorithm over a BEC to graphical objects called stopping sets. Another step in 
analyzing the performance of finite-length LDPC codes was taken by Richardson [7], where 
the performance of a given LDPC code, particularly in the error floor region, was related to the 
trapping sets of the underlying graph. Most recently. Cole et al. [8], devised an efficient algorithm 
for estimating the FER floors of soft-decision LDPC decoders on the AWGN channel using 
importance sampling. Estimations of the FER of GA and GB algorithms for certain categories 
of LDPC codes were also presented in [9], [10]. In an earlier paper [11], we derived upper 
and lower bounds on the FER of LDPC codes under hard-decision algorithms over a BSC. The 
bounds which were tight for sufficiently small crossover probabilities of the channel provided 
good estimates for the FER in this region. 

In this paper, we propose a method to estimate the performance of LDPC codes decoded by 
hard-decision iterative algorithms over a BSC. Compared to the methods of [7], [8], [9], [10] 
which first identify the most relevant trapping sets and then evaluate their contribution to the 
error floor, our approach does not require the identification of the trapping sets and is thus much 
simpler. Moreover, our approach is not only applicable to the error floor region, but also provides 
very accurate estimates in the waterfall region. In addition, we provide estimates for the BER, 
which is not even discussed in [7], [8], [9], [10]. In fact one of the important contributions of 
this work is to shed some light on the generally complex structure of the error events for iterative 
decoding, and to be able to establish simple approximate relationships between these complex 
events and the smallest initial error patterns that the decoder fails to correct. These relationships 
are the key in deriving the estimates. 

The rest of the paper is organized as follows. In section 11, we discuss different decoder 
failures for hard-decision algorithms. We then propose our FER and BER estimation methods 
and discuss their complexity in Section HI. Simulation results are given in Section IV. Section 
V concludes the paper. 
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II. Failures of Hard-Decision Iterative Decoding Algorithms 

Hard-decision iterative decoding algorithms are initiated by the outputs from the BSC. Hard 
messages are then passed between the variable nodes and the check nodes in the Tanner graph 
of the code through the edges of the graph in both directions and iteratively. We assume that 
the alphabet space for messages is {+1,-1} and that the decoder is symmetric [2]. To evaluate 
the performance, we can then assume that the all-one codeword is transmitted. 

An important category of hard-decision iterative algorithms are majority-based (MB'^) algo- 
rithms, where u is referred to as the order of the algorithm. For an MB'^ algorithm, in iteration 
zero, check nodes are inactive and variable nodes pass their channel messages. In iteration I > 1, 
check nodes pass the product of their extrinsic incoming messages in iteration l — l, and variable 
nodes pass their channel messages unless at least one-half plus to of their extrinsic incoming 
messages in iteration / disagree with the corresponding channel messages, in which case the 
negative of those (channel) messages are passed. In other words, for a variable node with degree 
dv and channel message mo, —mo is passed if at least [ci,;/2] +a; of extrinsic incoming messages 
are —mo, where [x] is the smallest integer which is greater than or equal to x. The order uj 
can be any non-negative integer less than or equal to d„ — 1 — [d„/2]. This maximum value 
corresponds to GA. 

Decoder failures of iterative algorithms can be related to graphical objects, called trapping 
sets [7]. A trapping set is defined as a set of variable nodes that cannot be eventually corrected 
by the decoder. In general, trapping sets depend not only on the structure of the graph but 
also on the decoder inputs (channel) and the decoding algorithm. For example, while for the 
belief propagation algorithm over the BEC, the trapping sets are precisely the stopping sets, 
for the maximum likelihood decoding over the BEC or any other channel, the trapping sets are 
the nonzero codewords. The structure of the trapping sets of hard-decision iterative decoding 
algorithms on a BSC is in general unknown. The following theorem identifies the structure of 
certain trapping sets for MB algorithms. This theorem is in fact a generalization of Facts 3 
and 4 of [7]. Facts 3 and 4 of [7] only apply to GA and GB on regular Tanner graphs with 
variable node degree 3 and check node degree 6. The following theorem extends this to irregular 
codes with arbitrary degree distributions where different nodes are allowed to perform different 
majority-based algorithms. 
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Theorem 1: For an LDPC code with degree distribution pair (A, p), suppose that variable 
node j performs MB of order ujj denoted by MB"^^ (0 < ujj < dj — 1 — \dj/2]), where dj is the 
degree of variable node j. Consider a set of variable nodes S and denote its induced subgraph 
in the code's Tanner graph by G{S). Also denote the check nodes which have an odd degree 
in G{S) by Co{S). If each variable nodecu j in the Tanner graph has at most \dj/2~\ + luj — 1 
neighbors from Co{S), then S is a trapping set. 

Proof: We show that if all the variable nodes in S are initially in error and all the other 
nodes are initially correct, then the decoder fails to correct the errors and the initial error pattern 
remains unchanged throughout the iterations. 

In the first iteration, the outgoing messages of a variable node j is the initial message mo G 
{— 1, 1} of node j. Based on the updating rule for MB algorithms in the check nodes, it is easy 
to see that node j receives —niQ along all the edges connected to Co{S) and tuq along all the 
edges connected to Co{S) = C\ Co{S), where C is the set of all the check nodes in the Tanner 
graph. This is regardless of whether node j is in the set S or not. Due to the structure of S, node 
j will therefore have at most \dj/2] +ujj — l incoming messages with their value equal to —mo. 
This implies that for each outgoing message of node j at the start of the second iteration, the 
number of extrinsic incoming messages with value —mo is also at most \dj/2] — 1. Hence 
the value of the outgoing message will remain mo. This is because, for the outgoing message 
to change, MB'^j requires at least \dj/2] + cuj extrinsic incoming messages with value —mo. 
This means that all the messages exchanged between variable nodes and check nodes, remain 
unchanged throughout the iterations. This implies that the decoded values of the variable nodes 
remain the same as the initial messages regardless of the number of iterations. ■ 
Theorem \T\ describes an instance of one type of trapping set which we refer to as fixed-pattern. 
For hard-decision algorithms in general, however, this is not the only type of trapping sets. 
In our experiments with different LDPC codes and different hard-decision iterative decoding 
algorithms, we have observed the following types of decoder failures corresponding to different 
types of trapping sets. The observations are made by tracking the error positions at the output 
of the decoder throughout iterations. 

1) Fixed-pattern: After a finite number of iterations, the error positions at the output of the 
decoder remain unchanged. 

2) Oscillatory-pattern: After a finite number of iterations, the error positions at the output of 
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the decoder oscillate periodically within a small set of variable nodes. 
3) Random-like: Error positions change with iterations in a seemingly random fashion. The 
errors seem to propagate in the Tanner graph and result in a larger number of errors at 
the output of the decoder even if the initial error pattern has only a small weight. 
Note that in the first two cases, there could be a transition phase, through which the error pattern 
settles into the steady-state. Also noteworthy is that in all the random-like cases that we have 
observed, the trapping set includes all the bits in the codeword, i.e., no bit can be eventually 
corrected by the decoder. In our experiments, we also observe that the percentage of random-like 
failures out of the total number of failures increases on average as the weight of the initial error 
patterns increases. 

The relationship between the trapping sets and the structure of the Tanner graph is in general 
complex. With the exception of Theorem [T] and code-specific results such as those of [9], [10], 
we are not aware of any other such result. Our experiments show that in many cases cycles in 
the Tanner graph are part of the trapping sets. The relationships among the cycles, the trapping 
sets and the initial error patterns that trap the decoder in the trapping sets, however, seem to be 
complex. For example, in a Tanner graph with girth 6, while variable nodes in cycles of length 
6 can form trapping sets for GA, they may be corrected by other MB algorithms. Moreover, 
there are cases where the variable nodes in a cycle of length 6 form a fixed-pattern trapping set 
for GA if these nodes are initially in error with all the other nodes received correctly. The same 
variable nodes, however, can be corrected by GA if the initial error pattern also contains other 
variable nodes. 

Due to the complexity of identifying and enumerating the different types of trapping sets for a 
given LDPC code under a given hard-decision iterative decoding algorithm, we take an approach 
different than that of [7], [8], [9], [10]. Our approach is universal in that it can be applied to 
any LDPC code with arbitrary degree distributions and to any decoding algorithm as long as the 
complexity of implementation is manageable. The algorithm is simply based on enumerating the 
initial error patterns of smallest weight that cannot be all corrected by the decoder. By using this 
information, we then estimate the contribution of all the other initial error patterns with larger 
weights to the total FER and BER. 
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III. Error Rate Estimation 

A. Frame Error Rate Estimation 

Consider a given LDPC code with block length n decoded by a given hard-decision iterative 
algorithm over a BSC with crossover probability e. Denote the set of all the error patterns of 
weight i by Si, and those that cannot be corrected by the decoder by Ei. Clearly, \Si\ = ("). 
Suppose that the decoder can correct all the error patterns of weight J — 1 and smaller, i.e., 
\Ei\ = 0, Vz < J. Also suppose that there are \Ej\ ^ weight-J error patterns that the decoder 
fails to correct. The FER is then equal to 

n " I z? I " 

FER = ^P(e|Op, = ^yp, = ^|E,|e^(l-er-\ (1) 

l=J l = J l=J 

where e is the event of having a frame (codeword) decoded erroneously, i denotes the weight 
of the initial error pattern at the input of the decoder, and pi is the probability of having i errors 
at the output of the channel (or the input of the decoder) given by the binomial distribution 

Pi= C^)£*(l-£)"-\ 

The first term of the summation in ([U) is denoted by P{J) and is equal to \Ej\e'\l — e)"""^. 
To estimate the FER, we enumerate Ej and calculate P{J) precisely. For the other terms in ([T]), 
we estimate \Ei\ as a function of \Ej\ as follows. 

For a given i > J, we partition the set Si as Si = S'- U S[, where S[ is the set of error 
patterns of weight i, each containing at least one element of Ej as its sub-pattern, and S[ is the 
complement set of S'^ in Si. The set S'^ thus contains the elements of Si that do not have any 
elements of Ej as their sub-patterns. We make the following assumptions: 

Assumption (a): No error pattern in the set S'^ can be corrected by the decoder. 

Assumption (b): Every error pattern in the set S^ can be corrected by the decoder. 

By the above assumptions, we have fl S*-] = l^-l and \Ein S[ \ =0, and therefore \Ei\ = 
\EinS[\ + \ EinS^\ = \S[\. The key in estimating \Ei\,i > J, is that IS*-] can be approximated as 
a function of \Ej\ using the following combinatorial arguments. Consider the probability V that 
a randomly selected weight-z error pattern contains at least one element of Ej as its sub-pattern. 
We then have 

P = ^ 1 - 1 - . (2) 



\Si\ V \s. 
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The first equality follows from the fact that there are 15^1 equally likely possibilities out of which 
\S[\ are the favorable cases. The second part of the equation is a consequence of approximating 
the random experiment by a sequence of (^) independent and identically distributed Bernoulli 
trials, each involving the selection of a weight-J error pattern with "success" defined as the 
pattern being in Ej. The probability of success is then ji^jl/ISjl, and the probability of having 
at least one success in (_^) trials is 1 — (1 — I^'jI/IS'jD^-^) following the binomial distribution. 



From Q, we have 

\Ej 



ISA ~ Si 



1-1 



15, 



\Ej 


(}) 




S 


J 





I^.IS^ = l^j|f^_^V (3) 



where the second approximation is obtained by considering only the first two terms in the 
binomial expansion. 

Note that Assumption (a) is valid for the belief propagation decoder on a BEC. For hard- 
decision iterative decoding algorithms on a BSC, although Assumptions (a) and (b) are not 
necessarily correct, they are approximately valid, especially for smaller values of i. In particular, 
our studies show that these assumptions are statistically viable and result in good approximations 

for l^il, i > J. 

Combining ([T]) and ([3]) with \Ei\ ^ \Sl\, i > J, we derive the following estimates for the 
FER: "Lower" estimate, 

N 



FER,{N) = P{J) + j^'il 



e)"-\ (4) 



i=J+l 

'Upper" estimate, 

TV 



i=J+l ^ [ ,;=0 



(5) 



where G {J + 1, J + 2, . . . ,n} is a parameter to be selected for the best accuracy of the 
estimates. It is clear that FERl{N) < FERu{N), and the equality holds if and only if A^ = n. 
The difference between the two estimates is the probability that the input error patterns to the 
decoder have a weight larger than A^. While FERl(N) is derived based on the assumption that 
error patterns of weight larger than A^ occur with negligible probability, FERu{N) is obtained 
based on the assumption that for such input error patterns the decoder fails with probability 
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one. Our observations show that in practice there exists a certain threshold Nq for error weights, 
around which a relatively abrupt change in the percentage of failures occurs. This is such that 
for error weights larger than Nq, the probability of failure goes to one very rapidly. 

Later in Section IV, we will see that FERu{Nq) provides a very accurate estimate of the 
FER for all channel crossover probabilities of interest. This suggests the following practical 
approach for determining Nq. We perform Monte Carlo simulations at high FER values, say 
around 0.01 — 0.1. We then choose A'o such that the estimate FERu^Nq) is the closest to 
the simulated FER. As the Monte Carlo simulations are performed at high FER values, their 
complexity is low and easily manageable. 



B. Bit Error Rate Estimation 

At the first glance, the problem of BER estimation may seem very complicated as we observe 
that even for the initial error patterns of the same weight, depending on the type of the trapping 
set, the number of bits in error at the end of the decoding can be quite different. Our further study 
into this however reveals that despite this variety of possibilities, one can estimate the average 
number of bit errors depending on the initial weight i of the error patterns. In general, we expect 
the conditional average BER given the initial error weight i to be an increasing function oi i. To 
simplify the analysis however, we partition the range of J < i < n into two subsets, J < i < Nq 
and Nq < i < n. For these partitions, we estimate the average number of bit errors by J and 
M, respectively, where M is the estimate of the average number of bit errors for error patterns 
of weight Nq, obtained by Monte Carlo simulations. We thus derive our estimate for the BER 
based on the FER estimate EERu^Nq), proposed in the previous subsection, as follows. 



BER ^ -PiJ) + -y \Ej\(''ri)s\l-er-^ + -\Ej\('''\)e''<^il 
n ^iijti \'^-'^J \Nq-JJ ^ 

No . X 



^\n-No 



M 
+ — 

n 



(6) 



In the following we provide the rationale behind the derivation of We recall that l-Ej] ~ 
\S[\. We partition S\ further as S[ = S'- U S'- , where S'- is the set of error patterns of weight i 
that have one and only one element of the set Ej as their sub-pattern. Using similar discussions 
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5:|.|S.|l'Vl-mV" m. (7) 



as those used for the derivation of ([3]), we have 

J J \ \Sj\ J \Sj\ 

It appears that for the error patterns of weight J < i < Nq, the ratio of 15-' 1/15" | is a large 
number and thus a large majority of the error patterns in the set Ei belong to the set S'- . This 
means that, for this range of error weights, many of the error patterns that result in decoding 
failures have one and only one element of the set Ej as their sub-pattern. For these error patterns 
we make the following assumption: 

Assumption (c): For the error patterns of weight J < i < Nq, the number of bit errors at the 
end of decoding is approximately J. 

Furthermore, we assume: 

Assumption (d): For the error patterns of weight Nq < i < n, the number of bit errors at the 
end of decoding is on average M, where M is the estimate of the average number of bit errors 
for error patterns of weight Nq. 

To determine M, we simulate a given number, say 10^ — 10^, of randomly generated error 
patterns of weight Nq. 

Our experiments show that both Assumptions (c) and (d) are statistically viable and provide 
very good estimates of the total BER. 

C. Computational Complexity 

To obtain an estimate of FER, one needs to enumerate J2i=i (T) ^^^^ patterns. For different 
error patterns, the iterative decoder performs the decoding and counts the number of decoding 
failures. Suppose that we are interested in estimating the performance of the LDPC code at a 
FER of p using Monte Carlo simulations and would like to observe at least m codeword errors 
for a reliable result. Assuming that the average number of computations required for iterative 
decoding in the two cases are the same, the ratio of the computational complexities of the Monte 
Carlo simulation and the proposed estimation method is: 



V = ^/P^^-Y (8) 

i=i ^ ^ 
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For BER estimation, we need to perform an extra number of iterative decodings to estimate 
M, and about 100m iterative decoding to obtain Nq. These are usually negligible compared to 

EtiC). 

It can be seen that r/ is a function of n, J, p and m. It increases with increasing m and with 
decreasing p, n and J. It often appears that J is a small number (< 4). For example, we have 
tested all the codes with rate 1/2 and n < 2048 in [12] under GA, and they all have J < 4. 
The value of m is often selected in the range of a few tens to a few hundreds. With given 
values for J, m and n, the proposed estimation method is more efficient than the Monte Carlo 
simulation (r^ > 1) if p < rn/J2i=i (")■ f^ct, for the block length and e values of interest, the 
computational complexity of the proposed method can be much smaller than that of the Monte 
Carlo simulations. 

One should also note that the value of r/ in ([8]) only reflects the saving in complexity compared 
to one Monte Carlo simulation point. Unlike our estimations that can be easily calculated for 
different values of e once we obtain \Ej\, in Monte Carlo simulations, for each simulation point, 
a new set of input vectors has to be generated and simulated. This makes the proposed method 
even more attractive from the complexity viewpoint. 

IV. Simulation Results 

To show that our method can be applied to both regular and irregular LDPC codes and to a 
variety of hard-decision iterative decoding algorithms, in this section, we perform experiments 
on four pairs of code/decoding algorithm. Code 1 is a (200, 100) irregular LDPC code. The 
degree distributions for this code, which are optimized for the BSC and GA, are given by 
X{x) = O.lllSx^ + 0.8885x3 ^^^^ ^ q_26x^ + 0.74x^ [3]. Code 2 is a (210, 35) regular 
LDPC code which has a variable node degree 5 and check node degree 6. For the degree 
distribution of this code, MB algorithm of order zero (MB°) has a better threshold than GA does 
[5]. Code 3 is a (1008, 504) regular LDPC code with variable node degree 3 and check node 
degree 6 taken from [12]. Code 4, which is a (1998, 1776) regular LDPC code taken from [12], 
has variable node degree 4 and check node degree 36. Tanner graphs for all the codes are free 
of cycles of length 4. Except Code 2, which is decoded by MB", all the codes are decoded by 
GA. In our simulations, the maximum number of iterations is 100 for all the decoders and for 
each crossover probability, we simulate until we obtain 100 codeword errors. 
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Table H] shows the values of J and \Ej\ for the four codes. It also shows the number and the 
percentages of different types of decoding failures for Ej. As can be seen, for all the codes, 
most of the decoding failures caused by initial error weight J have fixed patterns and there is 
no random-like failure. 

Figures [T] - 14] show the FER and BER performances of Codes 1-4, obtained by simulations, 
respectively. In these figures, we have also given the upper and the lower estimates of the FER 
for different values of N, as well as the estimates for the BER. For the estimates, we have 
used = 9, 13, 38 and 10, for Codes 1-4, respectively. The corresponding values of M 
are 7.73, 46.95, 143.93 and 133.57, respectively. From the figures, it can be seen that for each 
code, the lower estimate FERl{N) improves by increasing N. However, even the best estimate 
FERl^u) is only good at sufficiently small values of crossover probability. It fails to provide an 
accurate estimate at higher error rates. On the other hand, for all the codes, FERu(No) provides 
an impressively accurate estimate of the FER over the whole range of crossover probabilities 
of practical interest. It can also be seen that the BER estimates for all the codes follow the 
simulated BER curves very closely. 

To compare the computational complexity of our proposed method with that of the conven- 
tional Monte Carlo simulations, we consider the following example. 

Example 1: We consider estimating the performance of Code 1 at p = 10^"^. In this case, 
J = 3, n = 200 and we assume m = 100 as is the case for the simulation results presented 
in Fig. [TJ With these values, we have i] = 750. If we were to add another simulation point at 
p = 10~^, the complexity of Monte Carlo simulations would increase to 8250 times that of our 
proposed method. For the larger block length of 1008, our proposed method is more efficient 
than Monte Carlo simulations if the target FER p < 6 x 10^^. 

One should note that, as we discussed in Section III-C, for any given n and J, there exists a 
FER p', for which our method is more efficient than the Monte Carlo simulations in estimating 
the FER over the range p < p' . 

V. Conclusions 

In this paper, we propose a method to estimate the error rate performance of LDPC codes 
decoded by hard-decision iterative decoding algorithms over a BSC. By only enumerating the 
smallest weight (J) error patterns that cannot be all corrected by the decoder, the proposed 
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method estimates both FER and BER for a given LDPC code over the whole crossover probability 
region of interest with very good accuracy. The proposed method is universal in that it is 
applicable in principle to both regular and irregular LDPC codes of arbitrary degree distributions 
and to any hard-decision iterative algorithm. This universality is partly due to the fact that, 
unlike previous approaches, our method is not based on identifying the trapping sets and their 
relationship with the Tanner graph structure of the code and the decoding algorithm. 

Although the complexity of our proposed method is much less than that of the Monte Carlo 
simulations for many cases of interest, it still increases exponentially with J, essentially as n'^, 
where n is the block length. As a direction for future research, it would be interesting to find 
(to estimate) the number of error patterns of weight J that cannot be corrected by the decoder, 
using methods other than direct enumeration. 
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TABLE I 

Categorization of error Patterns with Weight J for Different LDPC Codes. 





Code 1 


Code 2 


Code 3 


Code 4 


J 


3 


4 


3 


3 


Ej 


1434 


18 


192 


200479 


Undetected Error 














Fixed-Pattern 


1303(90.87%) 


14(77.8%) 


165(85.9%) 


200438(99.98%) 


Oscillatory-Pattern 


131(9.13%) 


4(22.2%) 


27(14.1%) 


41(0.02%) 


Random-like 
















Fig. 1. PER and BER estimations and simulations of the (200, 100) irregular code (Code 1, \{x) = O.lllSa;^ + 0.8885a;^, 
p{x) = Q.2Qx^ + 0.74x^) decoded by GA. 
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Fig. 3. PER and BER estimations and simulations of the (1008, 504) regular code (Code 3, A(a;) = x^, p{x) — x^) decoded 
by GA. 



ffiEE TRANSCATIONS ON COMMUNICATIONS 



15 




